Prosthetic devices and methods and systems related thereto

ABSTRACT

Prosthetic devices, methods and systems are disclosed. Eye position and/or neural activity of a primate are recorded and combined. The combination signal is compared with a predetermined signal. The result of the comparison step is used to actuate the prosthetic device.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of provisional application60/636,679 for “Neural Prosthetic Combining Eye Position Information andNeural Activity” filed on Dec. 16, 2004, which is incorporated herein byreference in its entirety.

FEDERAL SUPPORT

This invention was made with U.S. Government support under DARPA grantno. MDA972-00-1-0029 and National Institute of Health grant nos. R01EY005522 and R01 EY013337. The U.S. Government has certain rights inthis invention.

BACKGROUND

1. Field

The present disclosure relates to control methods and systems applied toprosthetic devices and to methods and systems that incorporate and/orinvestigate neural bases of behavior.

2. Related Art

All publications herein are incorporated by reference to the same extentas if each individual publication or patent application was specificallyand individually indicated to be incorporated by reference. Thefollowing description includes information that may be useful inunderstanding the present invention. It is not an admission that any ofthe information provided herein is prior art or relevant to thepresently claimed invention, or that any publication specifically orimplicitly referenced is prior art.

Eye movements may be broadly categorized into those that are voluntaryand those that are involuntary. Among other things, involuntary eyemovements compensate for head movement; account for a moving backgroundbehind an object on which vision is focused; and act in a reflexivemanner to external stimuli. At least certain voluntary eye movements, onthe other hand, are known to relate to motor coordination and otherbehavioral attributes and processes.

Most voluntary eye movements are properly classified as saccades, assmooth pursuit eye movement, or as vergence movement. Saccades andsmooth pursuit eye movement relate to two dimensions in a visual field(i.e., the x- and y-axis in a coordinate system), while vergencemovement accounts for depth (i.e., the z-axis). More particularly,saccades are eye movements in which the eyes rapidly jump from one pointto another (e.g., from one word to the next while reading or around aroom when searching for an object); smooth pursuit eye movement involveseye movements that smoothly track slowly moving objects in the visualfield; and vergence movement—a relatively slow eye movement—occurs whenboth eyes coordinate to form an angle in a particular gaze direction(e.g., to focus on an object at a particular depth in the visual field).Voluntary eye movements act in concert with other physiologicalfunctions, such as motor function and psychological features ofperception, to coordinate behavior. Based on the coordinated nature ofbehavior, measurements of voluntary eye movement as a function of timeenable the prediction of movement.

Eyes move so quickly and easily that voluntary eye movements, generally,and saccadic eye movements, in particular, are a central feature ofprimates' natural behavior. Voluntary eye movements are not only crucialfor visual perception, but they also play an important role in motorcontrol and provide visual guidance for action. Indeed, orchestration ofhand and eye movements as we look and reach occurs frequently in naturalbehavior (D. H. Ballard et al., Spatio-temporal organization ofbehavior, Spatial Vision, 13:321-333 (2000); Land, M. F. & Hayhoe, M.,In what ways do eye movements contribute to everyday activities?, VisionRes., 41:3559-3565 (2001)). In addition to these sensory and motorroles, voluntary eye movements also participate in higher cognitiveprocesses. They are involved in shifting the locus of spatial attentionand both reflect and influence preferences and decisions (H. Scherbergeret al., Target selection for reaching and saccades share a similarbehavioral reference frame in the macaque, J. Neurophysiol.,89:1456-1466 (2003)). Studies of eye movements in humans undernaturalistic conditions reveal saccades are part of strategies to limitthe cognitive demands of a task (Land, M. F. & Hayhoe, M., In what waysdo eye movements contribute to everyday activities?, Vision Res.,41:3559-3565 (2001); M. M. Hayhoe et al., Visual memory and motorplanning in a natural task, J. Vis., 3:49-63 (2003); H. Scherberger etal., Target selection for reaching and saccades share a similarbehavioral reference frame in the macaque, J. Neurophysiol.,89:1456-1466 (2003)). Despite this multiplicity of roles in higher brainfunction, however, there has been relatively little physiological workstudying eye movements when the eyes are free to move. In fact, moststudies of eye movements have employed tasks with explicit instructionsthat require controlled fixation. While allowing a degree ofexperimental tractability, this approach is not well-suited forunderstanding voluntary eye movements, such as saccades, and theunderlying brain mechanisms during natural behaviors.

Another hallmark of natural behavior is decision-making. A body of worknow implicates a number of cortical areas in the neural basis ofdecision-makings; in particular, sensory-motor areas in the parietalcortex having strong anatomical connections with each other and withareas in the frontal cortex. Neuronal activity in these distributednetworks can be divided into two distinct classes: spiking and localfield potential (LFP) activity. Spiking is due to action potentials fromindividual cells while field potentials reflect synaptic activity andreturn currents from a population of cells near the tip of the recordingelectrode (U. Mitzdorf, Current source-density method and application incat cerebral cortex: investigation of evoked potentials and EEGphenomena, Physiol. Rev., 65:37-100 (1985)). Recent work studying areaLIP and PRR in the posterior parietal cortex shows that LFP activity aswell as spiking reflects information processing (Scherberger, H.,Jarvis, M. R., and Andersen, R. A., Cortical Local Field PotentialEncodes Movement Intentions in the Posterior Parietal Cortex, Neuron,46:347-354 (2005)). Despite the results showing that natural behaviorcritically depends on higher cortical function, there has been littledirect work on this at a physiological level.

Recent work in multiple institutions has demonstrated the feasibility ofa neural prosthetic based on cortical recordings. Some of this workfocused on decoding motor variables, such as movement trajectory (M. D.Serruya et al., Instant neural control of a movement signal, Nature,416:141-142 (2002); J. M. Carmena et al., Learning to control abrain-machine interface for reaching and grasping by primates, PlosBiol., 1:193-208 (2003); D. M. Taylor et al., Direct cortical control of3D neuroprosthetic devices, Science, 296:1829-1832 (2002)), while otherwork decodes cognitive variables such as movement goals and expectedvalue (S. Musallam et al., Cognitive control signals for neuralprosthetics, Science, 305:258-262 (2004)). But, whether coordinated eyemovements could also be used for this application was heretofore an openquestion.

There is therefore a need in the art for systems and methods thatincorporate measurements of eye movement—and particularly, voluntary eyemovement—in the mechanisms that control neural prosthetics, either aloneor in combination with cortical recordings relating to other functions,such as decision-making.

SUMMARY

The following embodiments and aspects thereof are described andillustrated in conjunction with systems and methods which are meant tobe exemplary and illustrative, not limiting in scope.

Subjects suitable for use in connection with various embodiments of theinvention include any animals that make eye movements, such as mammals,which, as used herein, refers to any member of the class Mammalia,including, without limitation, humans and non-human primates.

According to a first aspect, a method to control spatial positioning ofa prosthetic device of a subject is disclosed, the prosthetic devicebeing able to assume multiple spatial positions upon input from thesubject.

The method comprises: directly and/or indirectly recording eye positionof the subject to produce an eye position signal; recording neuralactivity relating to a motor, cognitive and/or other function of thesubject to produce a neural activity signal; combining the eye positionsignal and the neural activity signal to provide a recorded behavioralpattern; comparing the recorded behavioral pattern with at least onepredetermined behavioral pattern to identify a matching predeterminedbehavioral pattern, the at least one predetermined behavioral patternassociated with at least one predetermined spatial position of theprosthetic device; and positioning the prosthetic device in a spatialposition associated with the matching predetermined behavioral pattern.

According to a second aspect, a method to control spatial positioning ofa prosthetic device of a subject is disclosed, the prosthetic devicebeing able to assume multiple spatial positions upon input from thesubject.

The method comprises: directly and/or indirectly recording eye positionof the subject to produce an eye position signal; comparing the eyeposition signal with at least one predetermined eye position signal toidentify a matching predetermined eye position signal, the at least onepredetermined eye position signal associated with at least onepredetermined spatial position of the prosthetic device; and positioningthe prosthetic device in the spatial position associated with thematching predetermined eye position signal.

According to a third aspect, a prosthetic device able to assume aspatial position on input of a subject is disclosed, the prostheticdevice comprising: means for directly and/or indirectly recording eyeposition of the subject to produce an eye position signal; means forrecording the neural activity relating to a motor, cognitive and/orother function of the subject to produce a neural activity signal; meansfor combining the eye position signal and the neural activity signal toproduce a recorded behavioral pattern; means for storing at least onepredetermined behavioral pattern, the at least one predeterminedbehavioral pattern associated with at least one spatial positioning ofthe prosthetic device; means for comparing the recorded behavioralpattern and the at least one predetermined behavioral pattern toidentify a matching predetermined behavioral pattern; and means forpositioning the prosthetic device in a spatial position associated withthe matching predetermined behavioral pattern.

According to a fourth aspect, a prosthetic device able to assume aspatial position on input of a subject is disclosed, the prostheticdevice comprising: means for directly and/or indirectly recording eyeposition of the subject to produce an eye position signal; means forcomparing the eye position signal with at least one predetermined eyeposition signal to identify a matching predetermined eye positionsignal, the at least one predetermined eye position signal associatedwith at least one predetermined spatial position of the prostheticdevice; and means for positioning the prosthetic device in a spatialposition associated with the matching predetermined eye position signal.

According to a fifth aspect, a prosthetic device able to assume aspatial position on input of a subject is disclosed, the prostheticdevice comprising: an eye position recorder for directly and/orindirectly recording eye position of the subject to produce an eyeposition signal; a neural activity recorder for recording neuralactivity relating to a motor, cognitive and/or other function of thesubject and to produce a neural activity signal; a component forcombining the eye position signal and the neural activity signal toproduce a recorded behavioral pattern; a storage mechanism for storingat least one predetermined behavioral pattern, the at least onepredetermined behavioral pattern associated with at least one spatialpositioning of the prosthetic device; a processor for comparing therecorded behavioral pattern and the at least one predeterminedbehavioral pattern to identify a matching predetermined behavioralpattern; and a control system for positioning the prosthetic device in aspatial position associated with the matching predetermined behavioralpattern.

According to a sixth aspect, a prosthetic device able to assume aspatial position on input of a subject is disclosed, the prostheticdevice comprising: an eye position recorder for directly and/orindirectly recording eye position of the subject to produce an eyeposition signal; a processor for comparing the recorded behavioralpattern and at least one predetermined behavioral pattern to identify amatching predetermined behavioral pattern; and a control system forpositioning the prosthetic device in a spatial position associated withthe matching predetermined behavioral pattern.

According to a seventh aspect, a method to investigate neural basis of anatural behavior in subjects is disclosed. The method comprises:providing at least one subject; providing a behavioral task, thebehavioral task simulating a natural behavior in subjects; recording thebehavior of the at least one subject during the behavioral task;directly and/or indirectly recording the eye movement of the at leastone subject during the behavioral task; recording the neural activityrelating to a motor, cognitive and/or other function of the at least onesubject during the behavioral task; and combining the recorded behavior,the recorded eye movement and the recorded neural activity to identify apattern of recorded eye movement and neural activity characterizing thenatural behavior.

BRIEF DESCRIPTION OF THE DRAWINGS

The above-mentioned features and aspects of the present disclosure willbecome more apparent with reference to the following description takenin conjunction with the accompanying drawings wherein like referencenumerals denote like elements. It is intended that the embodiments andfigures disclosed herein are to be considered illustrative rather thanrestrictive.

FIG. 1 shows a schematic representation of a behavioral task inaccordance with an embodiment of the invention. Panel A shows sixsections, disposed along an arrow representing time flow during the taskand illustrating different moments of the task; in each section a monkeyis shown, reaching one of three visually identical targets (asterisks)located on a touch-screen in front of the monkey. Panel B shows a squaregrid illustrating the possible location of the targets in thetouch-screen of panel A, the square grid composed of eight posts spacedby 10° and disposed around the monkey's hand position (H) at the startof the task.

FIG. 2 shows a histogram illustrating the timing of behavioral eventsregistered in the task of FIG. 1 in accordance with an embodiment of theinvention. The x-axis shows the time in seconds; the y-axis shows thetarget reached by the monkey at the first second or third reach. Notethat second and third reaches were only made if the reward was notearned for the first reach.

FIG. 3 shows a schematic representation of the first reach choiceprobabilities for two monkeys subjected to the behavioral taskillustrated in FIG. 1 in accordance with an embodiment of the invention.The location of the circle shows the target position with respect to theinitial hand position (H). The darkness of the circle illustrates theprobability measured according to the scale reported as a bar on theright, wherein white color indicates 0 and the black color indicates0.5.

FIG. 4 shows a diagram reporting the saccade rate for a samplebehavioral session aligned to target onset in accordance with anembodiment of the invention. The x-axis shows the time in seconds; they-axis shows the saccade rate in Hertz.

FIG. 5 shows a diagrammatic representation of the eye position in atrial where two targets were present and one or the other was chosen inaccordance with an embodiment of the invention. Panel A shows locationof the two targets on a bi-dimensional plane with respect to the initialhand position (H); Panel B shows the eye position traces during the scanstage; and Panel C shows the eye position traces during the look-reachstage; traces are shaded according to which target was chosen.

FIG. 6 shows a histogram illustrating the viewing fraction to thepreferred (black bars) and non-preferred (white bars) target when thefirst reach was to the preferred and non-preferred target averaged overall pairs in accordance with an embodiment of the invention. The x-axisshows the reach choice; the y-axis shows the viewing fraction.

FIG. 7 shows a diagram illustrating the ROC analysis of viewing indexover time aligned to target onset or reach acquire averaged over allbehavioral sessions for monkey 1 (solid-line curve) and monkey 2(dashed-line curve) in accordance with an embodiment of the invention.

FIG. 8 shows the spike rasters and LFP spectrograms for the preferredand non-preferred directions registered in a monkey subjected to thebehavioral task of FIG. 1 before the first reach, in accordance with anembodiment of the invention. Regarding the LFP, the x-axis shows thetime expressed in seconds; the y-axis shows the frequency (Hz).

FIG. 9 shows a diagram illustrating the spike rate for the preferred andnon-preferred direction registered in a monkey subjected to thebehavioral task of FIG. 1 before the first reach in accordance with anembodiment of the invention. The x-axis shows the time expressed inseconds; the y-axis shows the rate in hertz.

FIG. 10 shows a diagram illustrating the 25-35 Hz LFP power for thepreferred and non-preferred directions registered in a monkey subjectedto the behavioral task of FIG. 1 before the first reach in accordancewith an embodiment of the invention. The x-axis shows the time expressedin seconds; the y-axis shows the LFP power expressed as μV²/Hz.

FIG. 11 shows a diagram illustrating the 1-10 Hz LFP power for thepreferred and non-preferred directions registered in a monkey subjectedto the behavioral task of FIG. 1 before the first reach in accordancewith an embodiment of the invention. The x-axis shows the time expressedin seconds; the y-axis shows the LFP power expressed as μV²/Hz.

FIG. 12 shows a diagram illustrating single subject choice probabilitiesfrom ROC analysis using spiking, 1-10 Hz LFP activity and 25-35 Hz LFPactivity in accordance with an embodiment of the invention. The x-axisshows the time expressed in seconds; the y-axis shows the choiceprobability.

FIG. 13 shows a diagram illustrating the population average choiceprobabilities from ROC analysis using spiking, 1-10 Hz LFP activity and25-35 Hz LFP activity in accordance with an embodiment of the invention.The x-axis shows the time expressed in seconds; the y-axis shows thechoice probability.

FIG. 14 shows a diagram illustrating a comparison of population averagechoice probabilities from ROC analysis using eye position and 25-35 HzLFP activity in accordance with an embodiment of the invention.Population average ROC analysis of eye position activity, 25-35 Hz LFPactivity and OC analysis of eye position and 25-35 HZ LFP activitycombined. The x-axis shows the time expressed in seconds; the y-axisshows the choice probability.

FIG. 15 shows a diagram illustrating the difference between choiceprobabilities during the scan stage with eye position and 25-35 Hz LFPactivity combined and with 25-35 Hz LFP activity alone and differenceexpected by chance in accordance with an embodiment of the invention.The x-axis shows the time expressed in seconds; the y-axis shows theprobability.

FIG. 16 shows a histogram illustrating the distribution of difference inchoice probability for OC analysis using 25-35 Hz LFP activity with andwithout shuffling eye position in accordance with an embodiment of theinvention. The x-axis shows the choice probability difference; they-axis shows number of recordings.

DETAILED DESCRIPTION

The systems and methods of the present invention are based on theincorporation of measurements of eye movement into, among other things,control mechanisms for neural prosthetics. As further described herein,measurements of eye movement or inferred eye movement can act as asurrogate for or supplement to other neural signals recognizingmovement. In various embodiments of the present invention, measurementsof eye movement may be combined with measurements of neural activityrelating to motor, cognitive, or other functions to enhance and/orrefine the control, operation, efficiency and/or accuracy of neuralprosthetics.

The various embodiments of this invention may be used with any animalsthat make eye movements, such as mammals, which, as used herein, refersto any member of the class Mammalia, including, without limitation,humans and non-human primates such as chimpanzees, and other apes andmonkey species; farm animals such as cattle, sheep, pigs, goats andhorses; domestic mammals such as dogs and cats; laboratory animalsincluding rodents such as mice, rats and guinea pigs, and the like.

Measurement of eye movement can be accomplished directly or indirectlyvia a number of techniques. In one embodiment of the present invention,eye movement may be directly measured with eye video tracking technology(e.g., using a bright light source to produce Purkinje images andtracking the relative movements of these images, and/or combining avideo image with computer software to calculate the position of thepupil and its center); with scieral search coils (i.e., small coils ofwire embedded in a contact lens are inserted into the eye such thathorizontal, vertical and/or tortional eye movement can be measured basedon an external magnetic field); and/or through a measurement of eyemuscle activity via electrooculography (i.e., measuring small voltagesrecorded from the region around the eyes that vary with change in eyeposition) or infrared oculography (i.e., directing an infrared lightsource at the eye and correlating eye position with the amount of lightreflected back to a fixed detector). Head position signals can be usedwith eye position signals to measure the direction of gaze when saccadesare produced by combined eye and head movements. Similar techniques tothose used for measuring eye position can be used for measuring headposition.

Detection of saccades, more specifically, can be performed by firstestablishing a velocity threshold (e.g., 50°/sec., 100°/sec., etc.).When the rate of eye movement reaches or exceeds this threshold, asaccadic event is determined to have occurred. Then, one can determinethe nature and extent of the saccade by identifying its beginning andend points. This may be accomplished by establishing a second velocitythreshold (e.g., 20°/sec.) that corresponds to the initiation andtermination of a saccade (i.e., when the eye velocity exceeds the secondvelocity threshold immediately prior to exceeding the first velocitythreshold, and when the eye velocity drops below the second velocitythreshold immediately following exceeding the first velocity threshold).

Alternatively or in addition to direct measurement of eye movement, eyemovement may be measured indirectly by a number of techniques inaccordance with an embodiment of the invention. Certain parts of thebrain produce neural signals relating to eye movement (e.g., the lateralintraparietal area of the intraparietal sulcus, the frontal eye fields,the supplementary eye fields, the superior colliculus, the medialparietal area, the dorsal and ventral prefrontal cortex, the dorsalpremotor cortex, the parietal reach region, etc.). Neural activitymeasured in one or more of these locations may be associated with eyemovement, by statistical inference or otherwise. This provides anindirect measurement of eye movement.

Still other parts of the brain produce neural signals that reflect eyeposition, although they are not themselves responsible for eye movement.By way of example, the visual cortex produces, among other things,signals relating to the intensity of light on the retina. Thus, if theposition of an external light source in the visual field is known, theneye movement may be indirectly measured based on the location on theretina of, for instance, greatest relative light intensity. It isconsidered to be within the scope of the present invention to implementsimilar, indirect techniques involving other areas of the brain andother correlations of brain function to eye movement.

Furthermore, neurons in many areas of the brain, including thosementioned above which carry saccade signals, also carry a signaldirectly related to eye position; that is the location of the eyes inthe orbits (R. A. Andersen et al., Neurons of area 7 activated by bothvisual stimuli and oculomotor behavior, Exp. Brain Res., 67:316-322(1987)). These signals may be derived from the integration of thesaccade command or by way of proprioceptive sensors in the eye muscles.These signals can be used to indicate the direction of the subject'sgaze. Other neurons carry both visual signals related to the location ofan object's projection on the retina and the position of the eyes in theorbits. These signals can add linearly or combine multiplicatively, or acombination of addition and multiplication. This interaction produces“gain fields” that can be used to determine the direction of gaze andthe location of targets with respect to the head (R. A. Andersen et al.,The encoding of spatial location by posterior parietal neurons, Science,230:456-458 (1985); Zipser, D., and Andersen, R. A., A back propagationprogrammed network that simulates response properties of a subset ofposterior parietal neurons, Nature, 331:679-684 (1988)). Other neuronscarry gaze direction signals or have gaze direction gain fields (P. R.Brotchie et al., Head position signals used by parietal neurons toencode locations of visual stimuli, Nature, 375:232-235 (1995)). Thegaze direction signal is a linear or quasi-linear sum of an eye positionsignal and a head position signal. All of the cases above can be used toextract eye and gaze direction for the described invention.

Thus, as used herein, and in connection with various embodiments of thepresent invention, measurements of “eye position” may take into accounteye position, head position, gaze direction, gain fields and/or other,similar features that relate to eye movement, position, and orientation.Therefore, measurements of eye and head movement as well as gaze andorbital direction may be integrated to provide a single measurement of“eye position.” Moreover, as used herein, an “eye position signal”produced by such measurements may thus include components of eyemovement, head movement, gaze, etc. The eye position signal, whichintegrates this variety of measurements relating to eye position, maythus be used to control or to supplement the control of a neuralprosthetic device as described in connection with alternate embodimentsof the present invention.

Any one or more of the aforementioned indirect measurements of eyemovement, head movement and gaze direction can be obtained usingconventional techniques, as will be readily appreciated by those ofskill in the art, such as spike measurements or measurements of LFP.Various devices and methodologies may be used to accomplish this featureof the invention, including, without limitation, the use of electrodes,optical measurements, and/or other mechanisms for detecting andquantifying brain function. These may be implemented in connection withcomputer software and/or other computational devices and machinery, asillustrated in the ensuing Examples.

Regarding spike measurements, in one embodiment of the presentinvention, an electrode or array of electrodes may be implanted into theregion of interest in a subject's brain and used to measure the signalsproduced by the firing of a single unit (SU) (i.e., a neuron) in thevicinity of an electrode. The SU signal may contain a high frequencycomponent. This component may contain spike-distinct events that exceeda threshold value for a certain amount of time (e.g., a millisecond).Spikes may be extracted from the signal and sorted using known spikesorting methods.

However, measuring SU activity with a chronic implant may be difficultbecause the SU signal may be difficult to isolate. An electrode may bein the vicinity of more than one neuron, and measuring the activity of atarget neuron may be affected by the activity of an adjacent neuron(s).The implant may shift position in the brain after implantation, therebychanging the proximity of an electrode to recorded neurons over time.Also, the sensitivity of a chronically implanted electrode to SUactivity may degrade over time.

Thus, in an alternate embodiment of the present invention, LFP may bemeasured. LFP is an extracellular measurement that represents theaggregate activity of a population of neurons. Information provided bythe temporal structure of the LFP of neural activity is believed tocorrelate to that provided by SU activity. Unlike SU activity, measuringLFP activity does not require isolating the activity of a SU.Accordingly, it may be advantageous to use LFP activity instead of, orin conjunction with, SU activity to obtain an indirect measurement of asubject's eye movement.

The activity of neurons in a subject's brain may be recorded with animplant. The implant may include an array of electrodes that measure theaction potential (SU) and/or extracellular potential (LFP) of cells intheir vicinity. In one embodiment, micro-electro-mechanical (MEMS)technology may be used to prepare a movable electrode array implant. Inalternate embodiments, the neural activity may be measured in formsother than electrical activity. These include, for example, optical orchemical changes, or changes in blood flow that may be measured bysuitable measuring devices.

Neural activity measured with an implant may be amplified in one or moreamplifier stages and digitized by an analog-to-digital converter. In anembodiment, multiple implants may be used. Recordings may be made frommultiple sites in a brain area, each of which conveys differentinformation. The signals recorded from different implants may beconveyed on multiple channels. By way of example, a system of theinvention may record signals relating to direct and/or indirectmeasurements of eye movement from one or more areas of the brainsimultaneously.

Therefore, spike measurement, measurement of LFP, or other knowntechniques such as the optical measure of neural activity using voltagesensitive dyes, metabolite sensitive (such as calcium release) dyes oractivity dependent blood flow changes, may be used to collect neuralsignaling information relating to eye movement, and, thus, to indirectlymeasure eye movement. This may be performed alone or in combination withdirect measurement of eye movement.

In one embodiment of the invention, a direct and/or indirect measurementof eye movement, either alone or in combination with head movement, gazedirection, gain fields, or the like, is used for various purposes, suchas the control of a neural prosthetic. In an alternate embodiment, adirect and/or indirect measurement of eye movement, either alone or incombination with head movement, gaze direction, gain fields, or thelike, is combined with a measurement of neural activity relating to oneor more motor, cognitive, or other functions to enhance and/or refinethe control, operation, efficiency and/or accuracy of a neuralprosthetic. Measurements pertaining to an array of motor, cognitive, orother functions may be so combined, in connection with alternateembodiments of the present invention, as will be readily appreciated bythose of skill in the art. While not wishing to be bound by anyparticular theory, it is believed that such a combination ofmeasurements more closely mimics the neural processing involved in braincontrol of normal motor function.

Measurement of motor, cognitive, or other functions can be accomplishedin any number of ways, such as by measurement of spiking and/or LFP incorresponding areas of the brain. For example, U.S. Pat. No. 6,952,687and U.S. patent application Ser. No. 11/086,534 each describe techniquesfor the measurement of cognitive control signals that are suitable foruse in connection with various embodiments of the present invention.Each of these references is incorporated herein in their entirety asthough fully set forth.

Therefore, according to a first aspect, a method to control spatialpositioning of a prosthetic device of a subject is disclosed, where theprosthetic device is able to assume multiple spatial positions uponinput from the subject.

The method comprises: directly or indirectly recording eye position ofthe subject to produce an eye position signal; recording neural activityrelating to a motor, cognitive and/or other function of the subject toproduce a neural activity signal; combining the eye position signal andthe neural activity signal to provide a recorded behavioral pattern;comparing the recorded behavioral pattern with at least onepredetermined behavioral pattern to identify a matching predeterminedbehavioral pattern, the at least one predetermined behavioral patternassociated with at least one predetermined spatial position of theprosthetic device; and positioning the prosthetic device in the spatialposition associated with the matching predetermined behavioral pattern.

In an alternative embodiment, the method comprises: directly and/orindirectly recording eye position of the subject to produce an eyeposition signal; comparing the eye position signal with at least onepredetermined eye position signal to identify a matching predeterminedeye position signal, the at least one predetermined eye position signalassociated with at least one predetermined spatial position of theprosthetic device; and positioning the prosthetic device in the spatialposition associated with the matching predetermined eye position signal.

Recording eye position can in particular be performed by any of theaforementioned techniques, whether direct or indirect. Recording neuralactivity can similarly be performed by any of the aforementionedtechniques, such as by detecting spike activity and/or LFP activity. Inparticular, the neural activity signal can be the spike rate and/or theLFP spectrum. In certain embodiments, the LFP activity recorded can bein the 25-35 Hz frequency band, although other frequency bands may besuitable for use in connection with alternate embodiments of the presentinvention. In one embodiment, the neural activity signal results fromcombining a separately recorded LFP spectrum and spike rate. Accordingto a different aspect, methods to control spatial positioning of theprosthetic device can be performed by detecting the eye position and/orneural activity only.

According to a further aspect, the invention includes a prostheticdevice that is able to assume a spatial position on input of a subject.The prosthetic device comprises: means for directly and/or indirectlyrecording eye position of the subject to produce an eye position signal;means for recording a neural activity related to motor, cognitive and/oranother function of the subject to produce a neural activity signal;means for combining the eye position signal and the neural activitysignal to produce a recorded behavioral pattern; means for storing atleast one predetermined behavioral pattern, the at least onepredetermined behavioral pattern associated with at least one spatialpositioning of the prosthetic device; means for comparing the recordedbehavioral pattern and the at least one predetermined behavioral patternto identify a matching predetermined behavioral pattern; and means forpositioning the prosthetic device in a spatial position associated withthe matching predetermined behavioral pattern.

In an alternate aspect, the invention includes a prosthetic device ableto assume a spatial position on input of a subject, in which theprosthetic device includes: means for directly and/or indirectlyrecording eye position of the subject to produce an eye position signal;means for comparing the eye position signal with at least onepredetermined eye position signal to identify a matching predeterminedeye position signal, the at least one predetermined eye position signalassociated with at least one predetermined spatial position of theprosthetic device; and means for positioning the prosthetic device in aspatial position associated with the matching predetermined eye positionsignal.

The means for recording eye position, for recording neural activity, forproducing the recorded behavioral pattern, for storing the behavioralpattern, for comparing the recorded and predetermined behavioralpatterns, and for positioning the prosthetic device may all be inelectronic communication with one another through any convenientconfiguration (whether hard-wired, wireless, or any combinationthereof), as will be readily appreciated by those of skill in the art.

There are many devices and techniques that may be used for recording eyeposition and neural activity, as described above.

Similarly, a recorded behavioral pattern may be produced through the useof a component that combines the eye position and neural activitysignals and implements a computational model and/or software algorithmto generate a behavioral pattern.

The behavioral pattern may be stored in any number of conventionalelectronic media, such as a hard disk drive (HDD), a compact disc (CD),a server (whether locally or remotely accessed), or any otherconventional storage mechanism that enables the storage of electroniccontent, whether fixed in the device of the present invention orremovable therefrom. Moreover, the format in which the behavioralpattern is stored may vary depending upon the particular features andintended use of the inventive device. For instance, behavioral patternsmay be stored in one or more databases, the architecture of which mayvery depending upon the computational features of the system. Variationswill be apparent to those of skill in the art, and can be optimized toaccount for parameters such as system efficiency, storage capacity, andthe like.

Recorded and predetermined behavioral patterns may be compared with aprocessor that has instructions to perform one or more particularcomparative tasks. These tasks may take the form of computer algorithms(e.g., described in software) that are configured to recognize patternsimilarities between and among behavioral patterns. Hash tables,machine-learning algorithms, and other commonly used techniques may bereadily implemented with a processor to achieve the goals of thisfeature of the invention. By way of example, U.S. Pat. No. 6,952,687describes a cognitive state machine configured to implement this type ofprocess, and the methods and systems described therein may be readilyused in connection with the present invention.

Control systems for positioning a neural prosthetic device are known inthe art, and any such system can be configured for use in connectionwith alternate embodiments of the present invention.

Further system components may also be included or integrated with theenumerated components, such as, for example, additional processors, amonitor, other peripheral devices, or like system components standardfor devices of this type.

According to a further aspect, a method and system are provided toinvestigate neural basis of a natural behavior in subjects, comprisingcombining a behavioral task that simulates natural behavior in alaboratory setting with recordings of eye movements and/or neuralactivity related to motor, cognitive and/or another function of thesubject. In the method at least one subject is employed. The subject maybe subjected to a behavioral task designed to simulate the naturalbehavior of interest. The behavior of the at least one subject duringthe behavioral task may be recorded, as well as the eye movement and/ora neural activity related to motor, cognitive and/or another function ofthe subject. The recorded behavior, the recorded eye movement and therecorded neural activity (in those embodiments in which neural activityis recorded) are then combined to identify a pattern of recorded eyemovement and/or neural activity characterizing the natural behavior.

In some embodiments, the task is designed to reduce the instructionsgiven to the subject letting the subject choose both where to look andwhere to reach. In those embodiments a pattern of behavior emergesnaturally as each animal learns a strategy for earning rewards mosteasily. The strategies shown by the subject in those embodiments are notcompletely idiosyncratic and contain common elements.

In one embodiment of the present invention, the aforementioned methodfor investigating a neural basis of a natural behavior in subjects isused as a “training” process for a neural prosthetic device. In thatregard, the training process is implemented with respect to the subjectuser of a neural prosthetic such that recorded eye movements, neuralactivity, and/or behaviors relate directly to the user, and cantherefore be used to calibrate the device. In alternate embodiments ofthe present invention, similar information collected from trainingprocesses with one or more other subjects can be used to calibrate thedevice for a user that has not participated in the training process.This may be particularly advantageous in those instances where theultimate user of a neural prosthetic is clinically incapable ofparticipating in the training process, due to, for example, paralysis orseverely compromised brain function; either or both of which theinventive neural prosthetic is designed to address.

The strong parallels existing between the results described herein andwork in behavioral game theory on human decision making which aims tounderstand human behavior in, for example, markets and auctions usinglab experiments (C. F. Camerer, Behavioral game theory: Experiments instrategic interaction, Princeton University Press, Princeton (2003))allow extension of results in humans.

Additional embodiments of the methods and systems can be envisioned by aperson skilled in the art upon reading of the present disclosure and inparticular the Examples section and will not be further described indetails.

The following examples are provided to describe the invention in furtherdetail. These examples, which set forth a preferred mode presentlycontemplated for carrying out the invention, are intended to illustrateand not to limit the invention.

EXAMPLES

The database for the experimental procedure illustrated in the examplescontained 44 behavioral recordings (35 from monkey 1; 9 from monkey 2),549 cell recordings (431 from monkey 1; 118 from monkey 2) and 1282field recordings (914 from monkey 1; 368 from monkey 2).

Example 1 Predicting Decisions from Eye Movements

To test eye movements and the neural basis of decision-making under morenatural conditions, monkeys were trained to perform a free choice taskinvolving a reach search between three targets. In particular, two malerhesus monkeys (Macaca Mulatta) participated in the experiments.

Experimental Preparation

Each animal was first implanted with a head cap and eye coil undergeneral anesthesia. In a second surgery 64 microelectrodes (2×32electrode arrays) were chronically implanted in the medial intraparietalarea (area MIP), part of PRR, in the posterior parietal cortex of eachanimal. Structural magnetic resonance imaging was used to identifyposition and orientation of the IPS and plan the entry point of theelectrode array on the cortical surface. Final placement of electrodearrays was determined visually during surgery after resecting the duraand exposing the IPS, parieto-occipital sulcus and post-central dimple.Implantation coordinates for the arrays were 6P,8L and 2P,12L, 4.5 mmbelow the cortical surface. Coordinates were determined using anelectrode manipulator mounted on a stereotaxic instrument (KopfInstruments, Tujunga, Calif.). Electrode arrays used sharp tungstenmicroelectrodes (Impedance 300 kf)) arranged in 4 rows of 8 electrodesspaced by 400 pm (MicroProbe, Inc., Potomac, Md.). An additional lowimpedance ground electrode was placed at each end of each array. Fulldetails of surgical procedures have been previously published (H.Scherberger et al., Magnetic resonance image-guided implantation ofchronic recording electrodes in the macaque intraparietal sulcus, J.Neurosci. Methods, 130:1-8 (2003)). All surgical and animal careprocedures were done in accordance with National Institute of Healthguidelines and were approved by the California Institute of TechnologyAnimal Care and Use Committee.

During each session multiple channels of neural activity were passedthrough a headstage (Plexon, Inc., Dallas, Tex.), filtered (1 Hz-10 kHz;custom), amplified (×10,000; TDT Electronics, Gainesville, Fla.),digitized (20 kHz; National Instruments, TX) and continuously recordedto disk for further analysis (custom C code). The high-pass filter onthe TDT amplifier had a significant roll-off resulting in an overallsuppression of power below 10 Hz.

Behavioral Task

Each monkey was trained to reach to targets presented on an LCD screenbehind a touchscreen (ELO Touchsystems, CA) for juice rewards accordingto a procedure schematically illustrated in FIG. 1. In a each of aseries of trials, monkeys were presented with three visually identicaltargets on a touchscreen in front of them. The three identical targetswere presented on a square grid spaced by 10°. Hand position at thestart of the trial was at the center of the screen. One target onlycontained a reward when touched and rewards were assigned in each trialwith equal probability. The monkeys made reaches to targets until theyearned the reward. The eyes were free to move and monitored using eithera sceral search coil (CNC Engineering, Seattle, Wash.) or, occasionally,an infrared video tracking system (Iscan Inc., Cambridge, Mass.).

In particular, the monkey started each trial by touching a green dot atthe center of the screen. After a baseline hold period (1000-1200 ms),the three visually-identical targets were presented on a 3×3 grid(spaced 10°) of eight possible locations around the start point. After ahold period (750-1000 ms) the monkey was allowed to reach to one of thethree targets. Only one of the three targets triggered a juice rewardwhen touched. If the monkey did not reach to the target that gave thereward, he was allowed to make additional reaches to targets followingsubsequent hold periods (500-750 ms). Additional reaches were permitteduntil the monkey received the reward. Targets were extinguished oncethey were touched. A different set of three targets from the eightpossible locations appeared each trial and the target that gave thereward was chosen from these three targets with equal probability. Thisstimulus-reward configuration set ensured the monkey didn't repeatedlyemit the same stereotyped sequence of movements. Reducing instructionsthe animal had to follow by allowing him to choose where to look andreach allowed us to simulate more natural behavior in a laboratorysetting. In support of this, each animal required relatively littletraining (1-2 weeks daily following initial chair training) before beingproficient (>90% correct) at the task. The timing of the behavioralevents registered during performance of the above behavioral task isillustrated on FIG. 2.

The registered reach choices, support the conclusion that each monkeyhad a strategy for choosing which target to reach to first. Eachstrategy was characterized by calculating the probability of reaching toa given target first. Other factors, such as hand used, were heldconstant, and the strategies were learned over several weeks after whichtime they became fairly stable.

Eye Movements Analysis

The strategies developed by the monkeys to reach the target and apossible predictive role of eye position was analyzed according to theexperimental approach illustrated in FIGS. 3 to 7.

In a first series of experiments, the probability of choosing eachtarget for the first reach by the two monkeys according to targetposition with respect to the initial hand position (H) was tested. Inparticular, the probability of reaching to a target was estimated bydividing the number of trials a target was chosen first, by the numberof trials that target was illuminated.

The results, illustrated in FIG. 3, show a common preference forchoosing ipsilateral targets closer to the initial hand position, sinceboth monkeys chose targets that were closer to the central start pointmore often than targets that were further away (monkey 1 made reacheswith his right hand and chose rightward targets with greater probabilityand monkey 2 made reaches with the left hand and chose leftward targets)(see FIG. 3). This common preference might be explained in terms ofminimizing the amount of physical effort required to get a reward.However, this explanation is reported for explanatory purposes only andmust not be intended to limit the purpose of the present disclosure. Intwo out of three trials animals made additional reaches to get a reward.The arrangement of targets around the starting hand position coupledwith biases in the first reach choice resulted in the second and thirdreaches being made in similar directions.

In a second series of experiments, the saccade rate for a samplebehavioral session aligned to target onset was tested. Saccades weredetected from eye position by smoothly estimating eye velocity andthresholding at 50°/s. Saccade rate was estimated by counting the numberof saccades that occurred in non-overlapping 50 ms windows throughoutthe trial. Trials were averaged aligned either to target onset or to theend of the first reach. The scan stage was 200-700 ms following targetonset. The look-reach stage was 300 ms before to 200 ms following reachstart.

The results illustrated in FIG. 4 show a striking pattern of hand-eyecoordination emerging as each monkey selected and executed the firstreach. Plotting the saccade rate during the trial revealed hand-eyecoordination was organized into distinct stages, and in particular inthe scan stage and look-reach stage (FIG. 4).

The scan stage occurs after the targets are presented. During this scanstage, the monkeys looked at the targets. In the scan stage, the saccaderate dipped before rising dramatically to a maximum of five saccades persecond (FIG. 4). The initial dip in saccade rate resulted from a changein the inter-saccadic interval distribution that was time-locked to theonset of the targets. This change has also been observed in humans (G.Ariff et al., A real-time state predictor in motor control: Study ofsaccadic eye movements during unseen reaching movements, J. Neurosci.,22:7721-7729 (2002)).

One possible explanation for the saccade rate dip of the scan stage isthat ongoing saccade plans were cancelled and restarted to reflect newtarget locations. This explanation is indicated for explanatory purposesonly and not intended to limit the scope of the present disclosure.

The look-reach stage occurs after the scan stage. During the look-reachstage the monkeys looked at the target and then reached to the target.In the look-reach stage the saccade rate dropped to a minimum and thenrose sharply as the monkeys made a coordinated hand-eye movement to thechosen target (FIG. 4).

In a third series of the experiments, the eye position was tested duringthe scan and the look-reach periods in trials where two targets werepresent and one or other was chosen.

The results illustrated in FIG. 5 show that the end-points of thesaccades contained information about the decision-making process. Inparticular, a saccade made to the target of the reach was registeredduring the look-reach stage. However, saccades made during the scanstage before this eye position indicated the target to be chosen despitethe presence of intervening saccades to other targets. This pattern waspresent across the database.

In a further series of experiments, the pattern detected in outcome ofthe experiments illustrated in FIG. 5, was quantified by calculating aviewing fraction during the scan period before the first reach andaveraging the viewing fraction across trials when the monkey chose onetarget or the other. The viewing fraction was estimated by calculatingthe fraction of time eye position was within a 5° window of given targetin a 200 ms window. According to this approach, the target in the pairthat the monkey chose more often was identified as the preferred target,and the other target as the non-preferred target. The time axis for allanalysis windows was aligned to the center of the window.

The results illustrated in FIG. 6 show a viewing fraction greater forthe preferred target than for the non-preferred target when the monkeychose the preferred target, and vice versa when the monkey chose thenon-preferred target (FIG. 6).

Additionally, while eye movements before the first reach tended to bemade toward the target to be chosen, eye movements later in the trialwere different. Specifically, in one animal, eye movements between thefirst and second reach were made to the target that would not be chosennext. This indicates eye movements were not only made for movementpreparation but also reflected a more abstract process related todecision-making.

In an additional series of experiments, the viewing fraction for eachtarget was converted into a single viewing index for each trial and usedthis index in a receiver-operating characteristic (ROC) analysis, tomeasure how much information eye position contained about the firstreach choice over time. The viewing index was computed for two targetsby taking the ratio of the difference between viewing fraction of eachtarget to the sum of both viewing fractions. This computation convertedthe viewing fractions for two targets into a single number suitable forROC analysis. ROC analysis measures the probability with which an idealobserver could discriminate between two alternatives given a set ofobservations. This probability, called a choice probability, has foundwidespread use in psychology and neuroscience. ROC analysis of all datawas done on a 200 ms window stepped by 50 ms through the trial beforeand during the first reach. Using a longer time window of up to 500 msincreased LFP spectrum choice probabilities but obscured the dynamics.The 200 ms window was chosen as it gave reasonable results and matchedthe inter-saccadic interval allowing analysis with respect to eyeposition.

Trials for ROC analysis were selected in which the same two targets werepresent and one was chosen. This procedure averaged over the location ofthe third target to give more trials for analysis. In some cases enoughdata was available to select trials in which the same three targets werepresent. However, this did not significantly change the resultsindicating the specific location of the third target had little effecton the activity in these data. At least 100 trials were available forROC analysis in each session. 95% confidence intervals were estimatedfrom the standard error of the mean choice probability. The eye positionon a 200 ms interval to match the average fixation interval during thescanning stage was measured and the ROC analysis of viewing index overtime aligned to target onset or reach acquire averaged over allbehavioral sessions for monkey.

The results illustrated in FIG. 7 show that the choice probability usingeye position was at chance level at the start of the trial and, onaverage, increased to 0.6 (monkey 1) or 0.7 (monkey 2) 250 ms after thetargets came on (FIG. 7). It then decreased before going to 1 at thetime of the look and reach. In one animal a significant increase inchoice probability was registered before the targets came on. Thisanimal had a tendency to view target locations that he was more likelyto choose even before targets were presented.

The above results show that eye position alone can predict reach choicesin advance of movement and that eye position is also a useful signal inthe study of decision-making.

Example 2 Predicting Decisions from Spiking and LFP Activity

The eye position and spiking and local field potential (LFP) activitywas recorded in monkeys during the behavioral task described in Example1.

In particular a series of experiments was run to see how neural activitydeveloped under the more natural conditions of the reach search task inwhich the monkeys were free to choose which target to reach to. Since aless-constrained behavior was tested, and there was no guarantee that acertain number of trials would be performed for each trial condition,the neural activity was recorded using chronically implantedmicroelectrodes. In particular, microelectrode arrays were implanted inthe medial intraparietal area (area MIP) of two monkeys and analyzedeach signal by predicting which target the animal chose to reach tofirst. This resulted in very stable recordings and allowed recordingmore trials from the same population of cells than we could withtraditional methods.

Spike events were extracted and classified from the broad-band activityusing open source (KlustaKwik; available from VA Software through theSourceForge® collaborative development environment) and custom Matlabcode (The Mathworks, Natick, Mass.). Usually only one cell was isolatedfrom each electrode.

LFP activity was estimated from broad-band activity by first medianfiltering the signal with a 1.5 ms filter and then low-pass filtering itat 300 Hz. Median filtering suppressed influence of spiking activity atlower frequencies. Only channels with large amplitude (>4 SD) spikingactivity were included in the spike database. All channels were includedin LFP database regardless of the presence of spiking activity on thatchannel. The LFP spectrum characterizes the amount of power in the LFPsignal at different frequencies. Analyzing LFP activity according tofrequency is important because different information is represented indifferent frequency bands.

Spike rate and LFP spectrum were estimated on a 200 ms window stepped by50 ms between estimates. The LFP spectrum was estimated with ±5 Hzresolution. ROC analysis of spike rate and LFP spectrum was done usingthese estimates. LFP power in each frequency band was analyzedseparately.

The spike rate and LFP spectrum were detected before the first reach fora pair of targets, a preferred target, which had higher overallactivity, and a non-preferred target.

The results referred to the spike rate, illustrated in FIG. 8 show thatthe spike rate built up to a maximum during the reach to the preferredtarget (FIG. 8), while before the reach to the non-preferred target,this buildup was not present, and the rate was even suppressed (FIG. 8).Additionally, spike rates started to show spatial tuning to the targetof a reach movement slowly after the onset of the targets (FIG. 9).

The results illustrated in FIG. 8 show that LFP spectrum revealedcomplex dynamics at different frequencies. At the start of the trial,there was a peak in the spectrum in a 20 Hz frequency band. After targetonset, power in this band was suppressed and the spectral peak shiftedto a higher frequency 25-35 Hz band. At the time of the reach, thispower was suppressed and activity shifted to a lower frequency 1-10 Hzband. A similar pattern could be seen for reaches to the preferred andnon-preferred targets. The LFP spectrum also showed spatial tuning tothe target of a reach movement, the spatial tuning developingimmediately after the onset of the target.

In particular, results illustrated in FIGS. 10 and 11 show a differentspatial tuning of the power to the target in the spectrum. Power in the25-35 Hz frequency band was spatially tuned after target onset throughthe reach. This tuning was absent during the reach tuning but itreturned after the reach. The 1-10 Hz frequency band only showed spatialtuning during the reach with an increase in activity for reaches to onedirection before reaches to the other. Across the database, 86% of cells(422 of 549), 62% (794 of 1282) of 2535 Hz LFP recordings and 93% (1192of 1282) of 1-10 Hz LFP recordings showed significant (p<0.01; ANOVA)spatially tuned activity during the task.

A comparison of the relative timing of spatial tuning in spiking and LFPactivity for these recordings with significant activity, suggested that25-35 Hz LFP activity became tuned earliest, followed by spike activityand then by 1-10 Hz LFP activity.

In view of the difficulties in directly comparing the strength of spikerates and LFP power quantitatively due to the statistically differentnature of these observations. Spiking activity is a point process ofspike times while LFP activity is a continuous process of voltagefluctuations. A receiver-operating characteristic (ROC) analysis wasthen used to address this problem. This analysis results in choiceprobabilities for either spike rate or LFP power that can be directlycompared. This makes ROC analysis well suited for the comparison ofspike rates and LFP spectra.

The results on a single subject illustrated in FIG. 12 show thatanalysis of the data on a 200 ms window choice probabilities using LFPpower in specific frequency bands were comparable with those based onspike rate but had different time courses during the trial.

A population average supported this result as illustrated in FIG. 13 andin agreement with our earlier analysis (above), LFP power between 25-35Hz was most strongly tuned early in the trial, increasing quickly toalmost 80%.

At the same time in the trial the choice probability using the firingrate of a single cell was much less, typically barely above chance, anddid not reach the same level for another 100 ms. Although the latency ofthe choice probability using spike firing was longer, we found itincreased to a maximum during reaching, often at levels of 100% forindividual cells. Interestingly, at this time choice probabilities basedon 25-35 Hz LFP power had decreased while those based on 1-10 Hz LFPpower had increased to approximately the same level as spiking.

Across the population of recordings with spatially-tuned activity thisprogression could also be seen examining the number of recordings withchoice probabilities greater than 0.6. At 300 ms after target onset, 23%(97 of 422) cell recordings, 70% (555 of 794) of 25-35 Hz LFPrecordings, and 12% (143 of 1192) of 1-10 Hz LFP recordings had choiceprobability greater than 0.6. In contrast, during reaching 86% (363 of422) cell recordings, 6% (48 of 794) 25-35 Hz LFP recordings, and 86%(1025 of 1192) 1-10 Hz LFP recordings had choice probability greaterthan 0.6. Since these comparisons were done based on choiceprobabilities, which have a common scale, the differences do not resultfrom a normalization bias. Instead, they result from a difference in thetiming of how information is represented in each signal.

While previous work shows spiking and LFP activity are coherent in somefrequency bands, these results show that spiking and LFP activity canalso be dissociated. This is similar to results in V1 during visualstimulation (N. K. Logothetis et al., Functional imaging of the monkeybrain, Nat. Neurosci., 2:555-62. (1999)). Specifically, the LFP activitywas found in a 25-35 Hz frequency band predicts reach choices beforespiking. This result could be due to a combination of factors such as abias in the cell population our recordings sampled and how activity isorganized in the neuronal network.

The finding of a lack of tuning for eye position in the 25-35 Hz LFPactivity is in counter-distinction to the strong tuning of spikingactivity by eye position. In PRR (which overlaps MIP) this tuningresults both from the retinotopic coding of response fields and directeye position modulation of these response fields.

The fact that these eye position effects do not appear to be present inMIP may reflect a lack of columnar organization for eye position.Alternatively, early LFP activity may be due to top-down inputs to thearea coming from reaching areas in the frontal lobe because activity inthese areas does not depend strongly on eye position (Cisek, P. &Kalaska, J. F., Modest gaze-related discharge modulation in monkeydorsal premotor cortex during a reaching task performed with freefixation, J. Neurophysiol., 88:1064-1072 (2002)). Spiking may insteadreflect outputs of the area. The presence of top-down inputs fromfrontal cortex to parietal cortex during free choice would haveimportant implications for how information is processed infrontal-parietal networks. These explanations are indicated forexplanatory purposes only and not intended to limit the scope of thepresent disclosure.

Example 3 Predicting Decisions by Combining Eye Position with NeuralActivity

To investigate whether combining eye position with neural activityimproves our ability to predict decisions for reaches and improve thecontrol of a neural prosthetic, a further series of experiments wascarried out.

To see whether combining eye position with neural activity would improvereach choice predictions, the choice probability using anoperating-characteristic (OC) analysis. OC analysis measures theprobability with which an observer can discriminate between twoalternatives given a set of simultaneously recorded signals (R. O. Dudaet al., Pattern Classification, 2nd Edition. New York: John Wiley andSons (2001)).

OC analysis of combined eye position and neural activity was done usingthe viewing index, firing rate and spectral estimates used for the ROCanalysis. OC analysis was performed by first ranking both viewing indexand neural activity estimates across trials for a pair of targets. Athreshold was then chosen for both quantities and a hit rate and falsepositive rate were computed for the threshold pair. This was repeatedfor different values of the threshold. An OC curve was generated byusing the minimum false positive rate for a given hit rate. The choiceprobability was the area under this curve. To test whether theperformance improvement of OC analysis of two variables wassignificantly better than ROC analysis of each variable alone, the datawere shuffled and estimated how much OC analysis of the shuffled dataimproved compared with ROC analysis of the same data.

This is similar to the ROC analysis used above for eye position, spikeand LFP activity individually in the experiments of Examples 1 and 2,but extends it to multiple signals.

The results illustrated in FIG. 14 show that a comparison of choiceprobability from an OC analysis combining eye position and 25-35 Hz LFPactivity with choice probability from an ROC analysis of eye positionand 25-35 Hz LFP activity, improved predictions.

These results indicate eye position and LFP activity containcomplementary not redundant information about decision-making.

Since according to the experimental procedures herein exemplified, eyeposition predicted reach choice perfectly at the time of the look reach,the choice probabilities earlier during the scan stage were furthertested.

The results illustrated in FIG. 15 show that during this stage combiningeye position and 25-35 Hz LFP activity improves choice probabilitiescompared with using either signal alone. In particular, choiceprobabilities during the scan stage immediately after target onset couldimprove by more than 25%. Similar results were obtained using spikingactivity. Since OC analysis uses more information more flexibly than ROCanalysis (see Methods) we would expect an improvement in choiceprobabilities using this technique by chance alone. To test whether theobserved choice probability improvements could be explained by chanceeye position and LFP activity were shuffled across all trials, thedegree of improvement for each session estimated and the distributionplotted (FIG. 15). The median choice probability improved by only 1.7%and the maximum by only 8% indicating the choice probabilityimprovements we observed were highly significant (p<0.001).

Example 4 Eye Position Tuning of LFP Plan Activity

The results of the ROC analysis also showed that eye position and LFPpower choice probabilities increase with similar latencies after theonset of the targets while those based on spiking activity have longerlatency.

A possible explanation of these differences can be that spiking and LFPactivity have different biophysical sources: Recorded spiking activityis biased toward the action potentials of larger neurons that are morelikely to be the outputs of an area while LFP activity reflects synapticactivity resulting from inputs to the area and local processing.Differences in the choice probability time course of these signals maybe a consequence of these different biophysical sources and how activityis organized in the neuronal network. Since neural activity in somebrain areas depends on eye position more than in others we reasoned thatstudying the dependence of LFP activity on eye position would helpanswer this question by teasing apart potential network sources of thissignal. For example, spiking activity in PRR, a likely site of ourrecordings, represents reach plans in an eye-centered reference frame(A. P Batista et al., Reach plans in eye centered coordinates, Science,285:257-260 (1999)). Spiking activity in eye movement area LIP also hasheavy projections to the site of the recordings (Lewis, J. W. & VanEssen, D. C., Corticocortical connections of visual, sensorimotor, andmultimodal processing areas in the parietal lobe of the macaque monkey,J. Comp. Neurol., 428:112-137 (2000)). In contrast, spiking activity indorsal premotor cortex, which also projects to the site of ourrecordings (P. B. Johnson et al., Cortical networks for visual reaching:physiological and anatomical organization of frontal and parietal lobearm regions, Cereb. Cortex, 6:102-19. (1996)), depends much less on eyeposition (Cisek, P. & Kalaska, J. F., Modest gaze-related dischargemodulation in monkey dorsal premotor cortex during a reaching taskperformed with free fixation, J. Neurophysiol., 88:1064-1072 (2002)).Therefore, whether or not LFP activity that is spatially tuned duringthe reach plan (LFP reach plan activity) depends on eye position hasnetwork implications.

Two methods were used to determine whether 25-35 Hz LFP reach planactivity depends on eye position. First, we tested whether there was adifference in LFP power for two different eye positions given the samereach choice. Only 29 of 794 sessions had a difference in their mean LFPactivity (p<0.05; t-test). This was not significant (permutation test:p>0.05). The relationship between LFP reach plan activity and eyeposition were changed by shuffling eye position across trials with thesame reach choice and repeated the OC analysis presented above, whereasshuffling should reduce the choice probability if LFP reach planactivity depends on eye position.

The results illustrated in FIG. 16 show that distribution of choiceprobabilities when eye position was shuffled was not different than whenthe relationship between eye position and the 25-35 Hz LFP reach planactivity was preserved (FIG. 16; p=0.85 KS-test). Therefore, LFP reachplan activity does not depend on eye position.

The results illustrated in the examples above support the conclusionthat before the first reach, eye position is predictive of reach choiceswhen the subject scans the targets. These results therefore indicate eyemovements participate in the reach selection process and support amethod for improving performance of neural prosthetics for paralyzedpatients that can still make eye movements.

The experimental procedures illustrated in the examples also support theconclusion that recorded spiking and LFP activity represent specific yetdifferent aspects of cognitive processing. In particular, LFP andspiking activity can be dissociated and are both predictive of reachchoices with LFP activity predicting reach choices before spiking. Theabove results therefore provide inter alia an approach to investigatingmultiple area spike-field recordings to understand the interactions ofneural activity across frontal-parietal networks.

Also since scanning eye movements predict reach choices earliest,followed by LFP activity and spiking activity in the medialintraparietal area (area MIP) combining eye position information andneural activity improved reach choice predictions early on.

In particular, the above results show that natural, unconstrainedhand-eye coordination contains a great deal of information about bothcognitive variables like decisions for plans during a scan stage andmotor variables like movement direction during a look-reach stage.Accordingly combining eye position information and neural activity canbe used in methods for improving performance of neural prosthetics forparalyzed patients that can still make eye movements included in thescope of the claims.

In particular, methods and prosthetic devices can be designed to recordeye position directly during prosthetic control and combine thisinformation with simultaneously recorded neural activity to improvepredictions.

Otherwise, a method or prosthetic device does not require directmeasurement of eye position. Instead it involves treating it as anunobserved variable whose value can be inferred from neural recordingsand behavioral recordings in the past. Neural recordings to infer eyeposition could be made from eye movement areas directly or from areasinvolved in coordinating the eye with other systems.

In either case, the above mentioned results suggest ways thearchitecture of neural prosthetic decoding schemes can be modified toimprove system performance by including eye position information

While the methods and systems have been described in terms of what arepresently considered to be the most practical and preferred embodiments,it is to be understood that the disclosure need not be limited to thedisclosed embodiments. It is intended to cover various modifications andsimilar arrangements included within the spirit and scope of the claims,the scope of which should be accorded the broadest interpretation so asto encompass all such modifications and similar structures. The presentdisclosure includes any and all embodiments of the claims.

1. A method to control spatial positioning of a prosthetic device, theprosthetic device able to assume multiple spatial positions upon inputfrom a subject, the method comprising: recording an eye position of thesubject to produce an eye position signal; recording a neural activityof the subject to produce a neural activity signal; combining the eyeposition signal and the neural activity signal to provide a recordedbehavioral pattern; comparing the recorded behavioral pattern with atleast one predetermined behavioral pattern to identify a matchingpredetermined behavioral pattern, the at least one predeterminedbehavioral pattern associated with at least one predetermined spatialposition of the prosthetic device; and positioning the prosthetic devicein the spatial position associated with the matching predeterminedbehavioral pattern.
 2. The method of claim 1, wherein the eye positionand the neural activity are simultaneously recorded.
 3. The method ofclaim 1, wherein the eye position is recorded directly.
 4. The method ofclaim 1, wherein the eye position is recorded indirectly.
 5. The methodof claim 1, wherein the neural activity is recorded by detecting spikeactivity, local field potential (LFP) activity, or both.
 6. The methodof claim 1, wherein the subject is a primate.
 7. A method to controlspatial positioning of a prosthetic device, the prosthetic device ableto assume multiple spatial positions upon input from a subject, themethod comprising: recording an eye position of the subject to producean eye position signal; comparing the eye position signal with at leastone predetermined eye position signal to identify a matchingpredetermined eye position signal, the at least one predetermined eyeposition signal associated with at least one predetermined spatialposition of the prosthetic device; and positioning the prosthetic devicein the spatial position associated with the matching predetermined eyeposition signal.
 8. The method of claim 7, wherein the eye position isrecorded indirectly.
 9. The method of claim 7, wherein the subject is aprimate.
 10. A prosthetic device able to assume a spatial position oninput of a subject, the prosthetic device comprising: means forrecording an eye position of the subject to produce an eye positionsignal; means for recording a neural activity of the subject to producea neural activity signal; means for combining the eye position signaland the neural activity signal to produce a recorded behavioral pattern;means for storing at least one predetermined behavioral pattern, the atleast one predetermined behavioral pattern associated with at least onespatial positioning of the prosthetic device; means for comparing therecorded behavioral pattern and the at least one predeterminedbehavioral pattern to identify a matching predetermined behavioralpattern; and means for positioning the prosthetic device in a spatialposition associated with the matching predetermined behavioral pattern.11. The prosthetic device of claim 10, wherein the means for recordingeye position is configured to record eye position directly.
 12. Theprosthetic device of claim 10, wherein the means for recording eyeposition is configured to record eye position indirectly.
 13. Theprosthetic device of claim 10, wherein the means for recording theneural activity is configured to detect spike activity, local fieldpotential (LFP) activity, or both.
 14. The prosthetic device of claim10, wherein the subject is a primate.
 15. A prosthetic device able toassume a spatial position on input of a subject, the prosthetic devicecomprising: an eye position recorder to produce an eye position signal;a neural activity recorder to produce a neural activity signal; acomponent to combine the eye position signal and the neural activitysignal to produce a recorded behavioral pattern; a storage mechanism tostore at least one predetermined behavioral pattern, the at least onepredetermined behavioral pattern associated with at least one spatialpositioning of the prosthetic device; a processor to compare therecorded behavioral pattern and the at least one predeterminedbehavioral pattern to identify a matching predetermined behavioralpattern; and a control system to position the prosthetic device in aspatial position associated with the matching predetermined behavioralpattern.
 16. The prosthetic device of claim 15, wherein the eye positionrecorder is configured to record eye position directly.
 17. Theprosthetic device of claim 15, wherein the eye position recorder isconfigured to record eye position indirectly.
 18. The prosthetic deviceof claim 15, wherein the neural activity recorder is configured todetect spike activity, local field potential (LFP) activity, or both.19. The prosthetic device of claim 15, wherein the subject is a primate.20. A prosthetic device able to assume a spatial position on input of asubject, the prosthetic device comprising: means for recording an eyeposition of the subject to produce an eye position signal; means forcomparing the eye position signal with at least one predetermined eyeposition signal to identify a matching predetermined eye position signalassociated with at least one predetermined spatial position of theprosthetic device; and means for positioning the prosthetic device in aspatial position associated with the matching predetermined eye positionsignal.
 21. The prosthetic device of claim 20, wherein the means forrecording eye position is configured to record eye position indirectly.22. The prosthetic device of claim 20, wherein the subject is a primate.23. A prosthetic device able to assume a spatial position on input of asubject, the prosthetic device comprising: an eye position recorder torecord an eye position of the subject to produce an eye position signal;a processor to compare the eye position signal with at least onepredetermined eye position signal to identify a matching predeterminedeye position signal associated with at least one predetermined spatialposition of the prosthetic device; and a control system to position theprosthetic device in a spatial position associated with the matchingpredetermined eye position signal.
 24. The prosthetic device of claim23, wherein the eye position recorder is configured to record eyeposition indirectly.
 25. The prosthetic device of claim 23, wherein thesubject is a primate.
 26. A method to investigate neural basis of anatural behavior in subjects, the method comprising: providing at leastone subject; providing a behavioral task, the behavioral task simulatingthe natural behavior in subjects; recording the behavior of the at leastone subject during the behavioral task; recording the eye movement ofthe at least one subject during the behavioral task; recording theneural activity of the at least one subject during the behavioral task;and combining the recorded behavior, the recorded eye movement and therecorded neural activity to identify a pattern of recorded eye movementand neural activity characterizing the natural behavior.
 27. The methodof claim 26, wherein the behavioral task is designed to minimizeexternal instruction and to allow the subject to choose where to lookand where to reach.
 28. The method of claim 26, wherein the subject is aprimate.